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Attorney Docket No. CP0005US 



METHOD AND APPARATUS FOR SCANNING ELECTRONIC 
5 BARCODES 

CROSS REFERENCE TO RELATED APPLICATIONS 
[0001] Not Applicable. 

10 

STATEMENT AS TO THE RIGHTS TO INVENTION MADE UNDER FEDERALLY 
SPONSORED RESEARCH AND DEVELOPMENT 
[0002] Not Applicable. 

1 5 FIELD OF THE INVENTION 

[0003] This invention relates to computer-readable codes such as barcodes, and more 

particularly to methods and apparatus for scanning codes from an electronic display or 
screen. 

20 BACKGROUND 

[0004] Barcodes provide a quick and convenient method of reliably entering 

information into a computer or computer system by scanning the barcode. They have 
become very common on a wide variety of consumer goods and services. An example of 
when a barcode is used with a service is when a barcode is printed on a membership card. 

25 Some barcodes are printed on packaging, such as barcodes indicating the universal product 
code ("UPC") of an item, and others are unique, such as a barcode printed on a parking 
garage ticket that indicates the date and time of when the ticket was taken. The data in the 
barcode is generally represented as a pattern of two levels of surface reflectivity, such as 
black and white stripes for a one-dimensional barcode. Two-dimensional barcodes also use a 

30 pattern of alternating reflectivities, but in a pattern of squares or other shapes. In either event, 
the data contained in the barcode is coded according to a binary coding system and the light 
and dark areas are interpreted as I's and O's or vice versa. 
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[0005] Barcodes are typically scanned by a scanner or reader (hereinafter *Yeader'). 

One type of reader has a lamp that uniformly illuminates the barcode, and an imaging sensor, 
such as a charge-coupled device ("CCD") array, CMOS detector, or other electronic imaging 
method, that detects the scattered light that is reflected off the barcode. Dark ink on light 

5 paper is a typical medium for presenting barcodes. The dark ink or paint provides low- 
reflectivity regions, and the light colored or matte (light scattering) paper provides high- 
reflectivity regions. Of course, one may use light paint on a dark background. 
[0006] Another type of reader builds a map of surface reflectivity by rastering a laser 

beam over the surface, A single photodiode detects the light reflected from each position of 

10 the laser spot on the surface. This type of barcode reader is called a laser barcode scanner. 
[0007] Unfortunately, neither approach is currently well-adapted to reading barcode 

images displayed on information display screens, such as the display screen of a wireless 
telephone ("cell phone") or personal digital assistant ("PDA"), Rendering a barcode on a 
display of a cell phone or PDA would be desirable because it would relieve the consumer 

1 5 from needing to carry the bits of paper usually associated with tickets and coupons, for 
example. 

[0008] Display screens can be divided into two classes, reflective and emissive. 

Reflective screens are broadly defined as screens that alter their reflectivity of ambient light 
to form an image, typically from light and dark pixels, such as passive black and white liquid 

20 crystal displays ("LCDs"). Emissive screens, such as backlit LCDs and cathode ray tubes 
(CRTs") internally generate the light emitted from their surface. Conventional laser barcode 
scanners do not work well with emissive display screens because the emissive displays do not 
necessarily change their surface reflectivity with image content. Imaging scanners may be 
able to read both types of screens, but might have difficulty with reflections off the surface of 

25 the display and insufficient contrast between the light and dark portions of the scannable 
code. 

[0009] One reason reflective display screens do not work well with conventional 

barcode readers is that the reflective display screens are typically designed with the 
assumption that little light will be incident from the direction of the viewer. In normal use 
30 the viewer's head will block light from this direction. Therefore, screens are often built 
without antireflection coatings on the front surface or at internal optical interfaces. 



[0010] However, the light source built into laser barcode scanners typically projects 

bright light from the direction of the camera or photodiode(s). This causes reflections in the 
image observed by the camera that can interfere with the displayed image. In addition, the 
overall light levels of barcodes displayed on reflective screens are different than for normal 
5 barcodes. For example, LCDs have internal polarizers that reflect a much smaller fraction of 
incident light than a painted or white paper surface. To the scanner, both light and dark 
pixels may appear dark. 

[001 1] Emissive screens may be easier for barcode scanners to read; however, they 

too may have compatibility problems. Emissive screens often have a diffusing front coating 

10 or surface treatment of the glass to reduce specular reflection of ambient light. Illuminating 
emissive screens with the lamps on a barcode scanner adds a scattered light background 
component to the light emitted from the screen. This background of scattered light reduces 
the contrast between light and dark pixels. Additionally, specular reflections of the light from 
the barcode scanner lamp to the camera or photodiode{s) can interfere with scanning as with 

1 5 reflective display screens. 

[0012] If a barcode reader can't scan the barcode the first time, it typically tries again 

and again. Often, the scanning fails and the clerk must take additional time to enter the data 
that would have otherwise been scanned into the computer. 

[00 1 3] Therefore, a barcode reader that can quickly and reliably scan codes rendered 

20 on an electronic display screen is desirable. It is further desirable that the barcode scanner be 
able to read codes from emissive or reflective screens. 

SUMMARY 

[00 14] A barcode reader uses an imaging technique to quickly and accurately scan 

25 barcodes rendered on electronic displays. A shade on the barcode reader blocks ambient light 
from the CCD imager ^'camera") and configures the barcode reader illumination to be off- 
normal from the surface of the display. A variety of CCD cameras are available from a 
variety of vendors, such as cameras sold under the trade name Megafill™ by Kodak. The 
light detector array is typically integrated with on-chip electronics for interfacing the camera 
30 to an instrument. In one embodiment, a CCD array with a 640 by 480 pixel detector array 
was used with a fill factor greater than 80%; however, it is possible to use cameras with lower 
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fill factors and fewer elements to read two-dimensional barcodes, especially if software 
filtering methods are used to enhance the image. The barcode reader can detect light emitted 
from the display to turn off scanner illumination, thus avoiding interference between the 
emitted light and the scanner light and specular reflections of the scanner light into the 
5 camera, 

[001 5] In a flirther embodiment, the exposure time of the CCD camera is set to 

between 2-20 times the vertical refresh period of a display. In a yet further embodiment, the 
refresh rate of the display is detected by the barcode reader and the exposure time of the CCD 
camera is adjusted accordingly to between 2-20 times the measured refresh rate. This 

1 0 enhances accuracy of the scan by reducing errors caused by the display flicker. 

[0016] The CCD imager uses automatic exposure or gain to optimize image contrast. 

In a particular embodiment, decoder software is used to set the detection threshold of the 
CCD camera. In an alternative or further embodiment, the scanner illumination is adjusted to 
achieve the desired contrast. In another embodiment, a CCD array with a wide spectral 

1 5 response is used to reduce or avoid Moire patterns from color pixels. 

[001 7] In another embodiment, the barcode scanner adjusts scaling in response to 

displays with non-square pixels. In a particular embodiment, an initial scan determines if a 
two-dimensional barcode is rendered as the standard pattern of squares, or as a pattern of 
rectangles arising from rectangular display pixels. The scanner alters the horizontal-vertical 

20 scaling to interpret the displayed rectangles as appropriate barcode elements. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0018] Fig. lA is a simplified cross section of a barcode reader according to an 

embodiment of the present invention applied to a display screen. 
25 [0019] Fig. IB is a simplified diagram illustrating the relationship between the 

illumination source, camera, and display screen. 

[0020] Fig. IC is a simplified diagram of a barcode scanning system according to an 

embodiment of the present invention. 

[0021] Fig. 2 A is a simplified representation of a black and white pixel. 

30 [0022] Fig. 2B is a simplified representation of a color pixel. 

[0023] Fig. 2C is a simplified representation of an image of the color pixel of Fig. 2B 



with a photopic camera. 

[0024] Fig. 3 A is a simplified flow chart of a process for detecting the type of display 

screen and setting a barcode scanner according to an embodiment of the present invention. 
[0025] Fig. 3B is a simplified flow chart of a process for setting exposure time 

5 according to display refresh rate according to an embodiment of the present invention, 
[0026] Fig. 3C is a simplified flow chart of a process for optimizing image contrast 

according to an embodiment of the present invention. 

[0027] Fig. 3D is a simplified flow chart of a process for scaling non-square scanned 

barcode elements according to an embodiment of the present invention. 
1 0 [0028] Fig. 3E is a simplified flow chart of a process for operating a barcode scanning 

system according to an embodiment of the present invention. 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
[0029] 1. Introduction 

1 5 [0030] The present invention provides a scanner for scanning computer-readable 

codes from an electronic display, such as two-dimensional barcodes. Scanners according to 
the present invention can incorporates both physical features, such alignment shrouds, 
illumination lamps, a high fill factor CCD camera with multiple pixel blocks for each barcode 
element, and a CCD camera with a wide spectral response for use with color displays. 

20 Scanning systems according to the present invention can also include a number of operating 
features, such as dynamically adjusting the exposure time to overcome display flicker or 
variations in brightness or reflectivity of the display screens, automatically re-scaling the 
scanning system to adapt to displays with non-square pixels, and identifying display type and 
configuring the scanner accordingly. 

25 

[003 1 ] 2 . An Exemplary Scanner 

[0032] Fig. 1 A is a simplified cross section of a barcode reader ("scanner") 10 

according to an embodiment of the present invention. The scanner is held against a display 
screen 12 that is illustrated as being flat, but could be curved. The display includes layers of 
30 glass or plastic 14, 16, which may be coated. For example, if the display is an LCD display, 
one of the layers would have a polarization coating. The scanner is sectioned along section 



line 17 to facilitate illustration of the interior 15 of the scanner, 

[0033] A CCD camera 18 is mounted in the scanner, which in this case includes a 

handle 20 with a trigger 21 that activates the scanner when depressed. A separate (from the 
imager) photodiode 13 is included to measure the emissivity of the screen, and the refresh 

5 rate in some embodiments. An illumination lamp 22 is mounted such that light emitted by 
the lamp is not directly reflected off of the layers of the display back into the CCD camera. 
In a particular embodiment, the lamp is placed so that the light from the lamp, represented by 
the arrows 26 is reflected off the outer layer 14 of the display at an oblique angle to avoid 
direct reflection into the CCD camera, as represented by the arrows 30. Generally, the lamp 

10 is placed on the center of the lateral axis above the CCD camera, but could be placed in a 
sideways oblique orientation. A cable 32 connects the scanner to a power supply and camera 
control and imaging electronics. A flat display is shown for simplicity of illustration. If a 
display is curved, it is typically only slightly curved, and not sufficiently to alter operation of 
the scanner. 

1 5 [0034] Some conventional scarmers have a built-in laser that generates a crosshair 

pattern over the imaged region to assist the user in aiming the scanner at the barcode. The 
image is intended to reflect off of a matte surface and does not show up well when projected 
onto some types of display screens, which can be both transparent and smooth. For reading 
displays, a mechanical alignment system, such as a wire frame attached to the scanner, should 

20 be used instead of the optical alignment marker. Alternatively, the CCD camera can be used 
to show the imaged region on a second computer display screen viewed by the sales clerk, so 
that the clerk sees what the scanner is imaging and adjust the position of the scanner 
accordingly. 

[0035] Scanners according to one embodiment of the present invention use an 

25 alignment shroud 34 that serves at least two functions. First, the sides of the shroud block 
ambient light (such as room lights) from entering the scanner and causing an error in reading 
the light level or causing bright specular reflections off the screen in the camera field of view. 
Second, the shroud aligns the CCD camera 18 and illumination lamp 22 relative to the 
surface 36 of the display screen 12 so that the bright light from the illumination lamp is not 
30 directly reflected into the CCD camera. Thus, the illumination lamp illuminates the display 
screen within the aperture of the CCD camera, but does not directly reflect into the camera. 



Otherwise, the result would be similar to pointing a video camera at a bright light. The 
specularly reflected light would saturate a portion of the camera field of view, preventing 
imaging of the barcode. The light from the illumination lamp should be incident to the 
surface of the display at an angle greater than the maximum angle measured from the edge of 
5 the imaged area nearest the illumination lamp. For example, if the camera viewing angle is 
between 10-13 degrees from normal to the screen, the light source should be more than 13 
degrees from normal to any point on the screen. 

[0036] The CCD camera is preferably a high fill factor CCD array with a fill factor of 

at least 80%. It is further preferable that the CCD camera has a wide spectral response, 

10 generally from about 400 nm to 700 nm in order to avoid Moire patterns that might otherwise 
form when scanning a color display screen that is generally designed to be viewed by a 
person. Moire patterns and similar optical artifacts can combine with the intended barcode 
pattern to cause scanning errors, and even a complete inability to recognize the barcode. 
[0037] It is particularly desirable that the sensitivity in the range of 400-700 nm not 

1 5 fall below 1 0% of the peak sensitivity within this range. Ideally, the detector would image 
the red, green, and blue pixels of a white display screen with equal sensitivity. This reduces 
the brightness variation across the complete pixel as measured by the camera. (The actual 
sensitivity to the 3 subpixel colors may vary slightly due to variations in subpixel spectra 
among different display types.) In a particular embodiment, a color filter between the 

20 detector array and the display screen compensates for wavelength-dependent sensitivit>^ of 
the detector. Specifically, the filter preferentially absorbs the middle (green portion) of the 
range compared to the ends (red and blue) since a photopic camera is more sensitive to the 
light emitted by the green subpixels of a typical display. It was found that near infra-red 
detectors used on conventional linear scanners did not work well because the display screen 

25 absorbed most of the light at the operating wavelength of the scanner. 

[0038] It is generally desirable that two-dimensional barcodes rendered to the 

electronic display use only square-format elements, such as two-dimensional barcodes 
according to the Datamatrix™ format or according to the Aztec™ format. Other barcode 
formats, such as PDF417™, which is a stacked linear (non-square) format, can also be used. 

30 For example, a two-dimensional barcode according to the Aztec™ format can be 15 by 15 up 
to 151 by 151 display elements (squares). 



[0039] When rendering a two-dimensional barcode to a display, it is generally 

desirable to display each barcode element as an NxM block of pixels to reduce the effect of 
Moire patterns, particularly on display screens with low fill factors. N and M are integers, 
and typically N^M; however, in some instance, such as with non-square display screen 

5 pixels, N might not equal M. Examples of portable electronic devices with displays having 
non-square pixels include portable high-definition television ("HDTV") and digital video 
disk ("DVD") players. Each element of the barcode is represented by one pixel, or by 2x2, 
3x3 or similar groups of pixels. Barcode standards typically require that all elements of the 
barcode be of equivalent size so arbitrary non-integer scaling of the barcode onto the display 

10 screen is not possible. However, the present invention overcomes this limitation by scaling 
the high-resolution image obtained by the CCD camera using post-imaging software. 
[0040] By making the barcode elements larger than one pixel, the feature of the pixels 

are made smaller relative to the barcode elements. This does not actually suppress 
interference, but makes the effects of interference easier to remove using filtering software, 

15 or to separate the imaged interference from the barcode features. For example, a "Gaussian 
blurr" or a two-dimensional Fourier Transform filter may be used to remove high spatial 
frequencies, such as would lie above the image artifacts associated with the pixels, while 
retaining the details of the barcode symbol. The filtering techniques may be adaptive, 
adjusting the filtering parameters to the interference pattern present in the image data. 

20 [0041] Fig. IB is a simplified diagram illustrating the relationship between the 

illumination source 18, camera 22, and display screen 12. For simplicity of illustration, it is 
assumed that the camera lens is relatively small so that it can be represented as a point, and 
that the screen is flat and perpendicular to the center axis 19 of the camera. Those skilled in 
the art will appreciate that this illustrative example can be adapted to screens that are slightly 

25 curved and to cameras with typical lens diameters, and to situations where the center axis of 
the camera is not precisely perpendicular to the screen. 

[0042] A barcode image 21 displayed on the screen is represented in side view as a 

solid dark region. The camera has a viewing angle (29 ) that is sufficient to image the entire 
barcode image. Ideally the viewing angle is slightly larger than the barcode image to 
30 facilitate alignment of the camera, as shown by the solid lines 27, 29. At a minimum it is the 
same as the size of the barcode as shown by the dashed lines 23, 25. Theta is typically 
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between 10-13 degrees for a distance d between the screen 12 and the camera lens of about 
15-20 cm and a width {2s) of the imaged region being about 14 cm. The general case is that 
the illumination source should be above a limit line 31 starting at the edge of the imaged 
region 33 and extending away from the screen at an angle from normal of inverse tangent 
5 is/2d). This configuration avoids specular reflections into the camera. 

[0043] 3. An Exemplary Scanning System 

[0044] Fig. IC is a simplified diagram of a barcode scanning system 40 according to 

an embodiment of the present invention, A scanner 10, such as is described above in 

1 0 reference to Fig. 1 A, is coupled to a computer 42 or computer network. The computer 

includes a processor 44 and memory 46, such as random-access memory ("RAM"), read-only 
memory ("ROM"), compact-disk ROM, magnetic hard drive, magnetic floppy drive, and 
other types of memory and combinations thereof as are generally known in the art. A 
computer-readable program 48 stored in the computer memory 46 configures the system to 

1 5 execute various tasks under the control of the processor 44. Additional input to the computer 
may optionally be provided with a user input device 50, such as a keypad or mouse used in 
conjunction with a screen, as is well known in the art. 

[0045] The scanning system is adapted in a number of fashions to scan barcodes from 

display screens. In one embodiment of the present invention, the CCD camera in the scanner 
20 10 detects whether any light is being emitted by the display, i.e, whether the display is an 
emissive display. This is done by starting with the illumination lamp off. If no emitted light 
is detected, then the system presumes that the display is a reflective display and the lamp is 
turned on. 

[0046] In another embodiment the system dynamically compensates for brightness 

25 variations across the display screen. For example, emissive screens often exhibit a brightness 
variation of up to 30% across the screen. This variation does not affect viewing by a typical 
user, as the human eye generally compensates for the variation. However, this variation can 
cause the light signals from light elements of one region of the barcode and dark elements of 
another region to be overlap in magnitude. This makes determination of light and dark 
30 elements by comparison with a threshold signal level during a scan inaccurate. 

[0047] The CCD camera is coupled through a cable 32 to control electronics 52 that 



can be adjusted by the computer according to instructions stored in memory and/or by 
dedicated memory, such as ROM in association with local processor, such as an application- 
specific integrated circuit. The control electronics can be integrated with the computer 
system, or coupled to the computer with a cable 54, as shown. The amplitude of the signal 

5 from detectors or detector blocks can be monitored to automatically adjust the gain in the 
control electronics to insure that the signals from a light barcode element are within a 
preselected range as the barcode is scanned. After the gain is adjusted to achieve a desired 
exposure range, the software can compensate for lighter or darker regions of the image 
arising from variations in the brightness or reflectance of a display. 

1 0 [0048] In the case of a reflective display, the reflectance and/or contrast can also vary 

across the screen, which can be compensated with software, as above for emissive displays, 
and/or the brightness of the illumination lamp can be adjusted. Thus, the reflected light will 
produce an output in a desired range. The illumination can also be adjusted according to the 
detected contrast between light and dark code elements, with or without associated gain 

15 control. 

[0049] Preferably, the gain of the CCD and the light level should be adjusted so that 

the brightest portion of the image does not saturate the detector. For example, if the detector 
has 8 bits of resolution, the detected brightness should be in the 0-255 range or a slightly 
smaller range. The image should not be too dim (e.g. all data in the 0-8 range) or low 
20 contrast {e.g. all data in the 5-16 range) or too bright {e.g. where many pixels would read at 
255). 

[0050] There are quite a few methods for adjusting the brightness, including adjusting 

the gain of the detector, the exposure time, the brightness of the illumination source (when 
scanning reflective displays) and the camera lens aperture, if such control is available. 

25 Contrast adjustment and compensation for brightness variations across an image can be 
adjusted using computer software methods. For example, each pixel could be compared to 
the average of all pixels out to some fixed distance from the pixel to determine if it is brighter 
or darker than the local average to determine if it is part of a dark or light barcode element. 
Barcodes are typically designed so that there are never large contiguous regions of all light or 

30 all dark pixels. Without threshold brightness adjustment across the scanned image, a barcode 
rendered on a non-uniform screen might have one end portion that is all light or all dark. 

10 



[005 1 ] Another feature of the scanning system is that it can adapt to non-square 

display pixels. The scanning system includes software that adjusts the resolution and scale of 
the scanned data to accommodate rectangular pixels. For example, although various two- 
dimensional barcode standards require square elements (within a specified tolerance), the 

5 reader software of the scanning system can accommodate non-square elements. The lack of 
squareness might arise from non-square pixels on the display, and/or image distortion, that 
might arise from a slightly curved screen or the scanner being held off-angle (not 
perpendicular) to the barcode image, friitially, the software steps would recognize and locate 
a barcode image. Then, the software would locate portions of the image that might represent 

10 barcode elements, and then measure the size of those portions. For example, Datamatrix™ 
barcodes have a particular border pattern. These measurements are then used to identify the 
individual element locations of the barcode within the image, correcting for any distortions. 
It is possible that the maximum allowable distortion tolerance specification may be 
broadened compared to corresponding printed (paper) barcodes. 

15 

[0052] 5 . Scanning of Color Pixels 

[0053] In a fiirther embodiment, the scanning system fiirther corrects for differences 

between black and white displays and color displays. Referring to Fig. 2A, in black and 
white displays a pixel 80 typically consists of a square active region 82 surrounded by a 
20 border 84. For typical LCDs and CRTs the active region exceeds 50% of the pixel area, and 
can be as high as 90% in some displays. However, in color LCD screens pixels are divided 
vertically into three colored subpixels that combined make up about 30-60% of the pixel area. 
The subpixels emit or reflect red, green, and blue light, which are mixed to display other 
colors. 

25 [0054] In CRTs the red, green, and blue regions may be small dots or stripes of 

colored phosphor. There may be more than one dot of each color in each complete subpixel. 
When the colored elements of the subpixel are imaged with the monochrome CCD camera in 
the scanner, the colors will appear as different shades of gray. Black and white 
(monochrome) cameras are often designed to mimic the color response of the human eye. 

30 These cameras are called photopic cameras. Imaged with a photopic camera, the green 
components of a white color subpixel will appear brighter than the red and blue components, 
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Some cameras are more sensitive to near-infrared light and will image the red portion as 
brightest. 

[0055] Fig. 2B illustrates the internal structure of a white-colored pixel 86 on a 

backlit color screen. The pixel includes a red stripe 88, a green stripe 90, and a blue stripe 

5 92. Fig. 2C illustrates an image 94 of the white-colored pixel shown in Fig. 2B as imaged 
with a photopic black and white camera. All subpixel images are shades of gray, the imaged 
green subpixel 90' being the brightest, and the imaged blue subpixel 92' being the darkest. 
The imaged color pixel generally has reduced brightness, thus reducing contrast between the 
light and dark code elements. In a preferred embodiment, the CCD camera is not a photopic 

10 camera, but has spectral range similar to the human eye with more constant sensitivity 
throughout the spectral range. Alternatively, an optical filter that preferentially transmits red 
and blue light compared to green light can be used to balance the effective spectral response 
of a photopic camera when used with the filter. 

[0056] Moire patterns are another problem that can arise if the camera (detector) and 

1 5 display pixel active regions do not make up the entire image sensor area and display, 
respectively. Moire patterns, which are large periodic variations in brightness, can be 
generated in the image due to interference between the two (display and sensor) pixel arrays. 
There are several methods by which this interference can be minimized. First, the camera 
pixels should cover as great a fraction of the image area as possible. This makes the output 

20 of the camera pixel an average measurement of the greatest possible area of the display area. 
Second, if size allows, the barcode elements should be made up of two or more of the display 
pixels. This puts the pixel substructure on a different spatial scale from the barcode structure, 
as discussed above in paragraph 41 . Third, the spectral response of the camera should be 
chosen so that it images red, green, and blue subpixels with roughly equal sensitivity so that 

25 the colored subpixels are uniformly imaged. Finally, the barcode reader software may 
include adaptive low-pass filtering that removes spatial features from the image that are 
smaller than the size of the barcode elements. Many examples of this software are known in 
the art, such as Gaussian blurr filtering and two-dimensional Fourier transform filtering. 
Some implementations of such software can be adaptive, adjusting its filtering parameters to 

30 maximize the removal of the interference pattern. 

[0057] In addition to the spatial interference between the pixel structure of the display 

12 



and the camera, there may be temporal interference. Display screens are typically refreshed, 
or the image is re-displayed, at between 25-100 Hz. As a result, some screens, notably CRTs, 
flicker at the refresh rate. LCDs also flicker somewhat, but the relatively slow response 
(persistence) of the liquid crystals filters out most of the variation in brightness arising from 
5 the refresh cycle. Likewise, the cameras periodically sample and output an image. The 
sampling rate and refresh rate can interfere to cause severe brightness variations in the 
barcode image. 

[0058] There are at least two methods to prevent this effect. One is to reduce the 

sample rate of the camera so that it will always include at least several complete refresh 

10 cycles. Ideally, the camera should capture 2 or more, preferably 10 or more, and more 
preferably 10-20 refresh cycles in each imaging cycle. For example, if the refresh rate is 25 
Hz, the exposure time would be at least about 0.08 seconds, preferably 0.4-0.8 seconds, and 
Using this technique, the brightness modulation of the display refresh rate is averaged out. 
Unfortunately, this means capturing images at a slow rate. The image can smear if the user 

1 5 does not hold the scanner and displayed code steady; however, the refresh rate on many non- 
interlaced computer displays is at least 60 Hz, thus an exposure time of about 0.17-0.33 
seconds would be appropriate for use with these systems, 

[0059] An alternative approach is to synchronize the imaging camera to the display 

refresh rate. The refresh rate can be detected with a photodiode and used to synchronize a 

20 phase-locked-loop ("PLL") frequency generator inside the scanner. A photodiode in the 
scanner can detect the refresh rate by providing a signal corresponding to the least amount of 
detected light, which occurs during the vertical blanking interval for a CRT. The period of 
this signal is the refresh period. The scanner exposure time per image can then be set to be an 
integer multiple of the refresh period. With the refresh and exposure time periods locked 

25 there will be no temporal interference between the camera and the display. The camera 
shutter could be synchronized to open and close during this period with an integer number of 
vertical blanking intervals in between. This provides an accurate scan from a single exposure 
once the frequencies are locked. 

30 [0060] 4. Exemplary Processes 

[006 1 ] Fig. 3 A is a simplified flow chart of a process for detecting a type of display 
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screen and setting the illumination of a barcode scanner 300 according to an embodiment of 
the present invention. The process is typically implemented according to a computer 
program (see, e.g. Fig. IC, ref. num. 48) having a number of computer-executable 
instructions stored in a computer-readable memory or medium. The computer-readable 
5 medium might be integrated into a scanning system, or might be separate from the system, 
such as a diskette or CD-ROM, for loading ("reading") into the system, or might be stored in 
a remote medium, such as a server, and dow^nloaded to the scanning system. The scanning 
system can thus be configured according to the computer program to perform the process. 
[0062] A scanner with an imaging camera is placed against the display (step 302) and 
10 the light output from the display screen is measured (step 304). The scanner includes an 
illumination lamp that is initially off, and preferably includes a shroud to exclude ambient 
light from entering the camera and causing an error in reading the light level. The light can 
be measured w^ith a photodiode (peak brightness over a period at least as long as the slowest 
refresh rate), or by imaging the screen with the camera, for example. The light output is 
1 5 then evaluated to determine if the screen is bright or dark by comparing the measured light 
to a programmed threshold value (step 306), If the measured light exceeds the threshold 
value, the display is presumed to be emissive and the scanner scans the displayed barcode 
(step 308) without turning on the illumination lamp. Leaving the illumination lamp off 
improves the first scan rate (FSR) by improving the contrast, or more specifically, not 
20 degrading the contrast, of the displayed barcode. If the display is not emissive, the 

illumination lamp is turned on (step 310) before scanning the displayed barcode (step 308). 
[0063] Fig. 3B is a simplified flow chart of a process for setting the exposure time of 

a barcode scanner according to the refresh rate of an emissive display screen 320 according to 
another embodiment of the present invention. The process is typically implemented 
25 according to a computer program (see, e.g. Fig. IC, ref. num. 48) having a number of 
computer-executable instructions stored in a computer-readable memory or medium. The 
computer-readable medium might be integrated into a scanning system, or might be separate 
from the system, such as a diskette or CD-ROM, for loading ("reading") into the system, or 
might be stored in a remote medium, such as a server, and downloaded to the scanning 
30 system. The scanning system can thus be configured according to the computer program to 
perform the process. 
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[0064] A preliminary image is captured by the camera (step 322) and automatically 

evaluated by the computer according to instructions stored in software to determine if the 
image data represents over or under exposure (step 324). If the exposure is incorrect, the 
camera gain or the exposure time is adjusted (step 325) and another image is captured (step 
5 326). If the exposure appears correct, decoding of the barcode image is attempted (step 327). 
If the barcode image can not be decoded, the display screen is tested for screen flicker by 
measuring the brightness with the photodiode for a sufficient period of time to measure the 
refresh rate of the display (step 328), typically for about 50-200 ms. If the barcode image is 
successfully decoded, an indication of a successful reading is provided (step 329). 
1 0 [0065] If the screen is not flickering above a programmed threshold (Le, the minimum 

detected level is not sufficiently different from the maximum detected level), a barcode read 
failure is indicated (step 330). If flickering is detected, the flicker frequency and phase is 
determined (step 332) from the data measured in step 328. An exposure time that is an 
integer multiple of the flicker period is set (step 334), and an image is captured using the 
15 exposure time and phase determined in step 332 (step 336). An attempt to decode this 

barcode image is made (step 338), and indication is made if the decoding was successful (step 
329). Indications of successful and unsuccessful readings can be different audible tones, 
different lights on the scanner viewed by the operator, or messages written to a display of the 
scanner system, for example. 
20 [0066] Fig. 3C is a simplified flow chart of a process for optimizing image contrast of 

a reflective display screen 340 according to yet another embodiment of the present invention. 
The process is typically implemented according to a computer program {see, e.g. Fig. IC, ref. 
num. 48) having a number of computer-executable instructions stored in a computer-readable 
memory or medium. The computer-readable medium might be integrated into a scanning 
25 system, or might be separate from the system, such as a diskette or CD-ROM, for loading 
("reading") into the system, or might be stored in a remote medium, such as a server, and 
downloaded to the scanning system. The scanning system can thus be configured according 
to the computer program to perform the process. 

[0067] A scanner with an imaging camera is placed against a display screen showing 

30 a barcode (step 342). The illumination lamp is already on, or is turned on (step 344), and an 
image of the displayed barcode is captured with the camera (step 346). The image data is 
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evaluated for over or under exposure (step 348), and if the exposure is correct, an attempt is 
made to decode the barcode image (step 350). If the barcode image is successfully decoded, 
a signal indicating a successful barcode reading is provided (step 352). 
[0068] If the image is not correctly exposed, the exposure time, illumination lamp 

5 brightness, aperture, and/or camera gain is adjusted (step 354) according to the data evaluated 
in step 348. Once the exposure parameter(s) has been adjusted, another image is captured 
(step 356), and an attempt is made to decode this image (step 350). If decoding is successful, 
the signal indicating a successful decode is provided (step 352), if not a signal indicating an 
unsuccessful scan is provided (358). Additional loops may be added to the process to attempt 
10 to correct exposure a selected number of times before indicating a scan failure. 

[0069] Fig. 3D is a simplified flow chart of a process for scaling non-square scanned 

barcode elements 360 according to another embodiment of the present invention. The 
process is typically implemented according to a computer program {see, e.g. Fig. IC, ref 
num. 48) having a number of computer-executable instructions stored in a computer-readable 
15 memory or medium. The computer-readable medium might be integrated into a scanning 
system, or might be separate from the system, such as a diskette or CD-ROM, for loading 
("reading") into the system, or might be stored in a remote medium, such as a server, and 
downloaded to the scanning system. The scanning system can thus be configured according 
to the computer program to perform the process. 
20 [0070] An imaging scanner captures an image of a barcode displayed on an electronic 

screen (step 362). Typically, the detectors in the camera array output an analog signal 
proportional to the intensity of the light illuminating that detector. This analog output is 
typically amplified, with the gain being adjusted for intensity and contrast, and digitized on 
the camera chip. Hence, the output of the imaging camera is typically in digital form. 
25 [007 1 ] The image is provided to a digital processor (step 364) that evaluates light and 

dark regions of the image (step 366) according to instructions from a computer-readable 
program to determine if the light and dark regions corresponding to tw^o-dimensional barcode 
elements are square by comparing the lengths of adjoining sides of the regions. If the regions 
are not square, the processor scales the image, either by lengthening one axis or /and 
30 shortening the other to translate the image into a scaled virtual image of square elements 
according to the measured ratio of the sides (step 368). The scaled virtual image is then 
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electronically processed ("scanned") to read the barcode represented by the scaled virtual 
image (step 370). 

[0072] Fig. 3E is a simplified flow chart of a process for operating a barcode scanning 

system 380 to scan barcodes rendered on an electronic display screen according to an 

5 embodiment of the present invention. The process is typically implemented according to a 
computer program {see, e.g. Fig. IC, ref. num. 48) having a number of computer-executable 
instructions stored in a computer-readable memory or medium. The computer-readable 
medium might be integrated into a scanning system, or might be separate from the system, 
such as a diskette or CD-ROM, for loading ("reading'^) into the system, or might be stored in 

10 a remote medium, such as a server, and downloaded to the scanning system. The scanning 
system can thus be configured according to the computer program to perform the process. 
[0073] The scanner is placed against an electronic screen displaying a barcode (step 

381) and the screen type is evaluated (step 382) with a photodiode to determine if the screen 
is emissive or reflective. If the screen is reflective, the illumination lamp is turned on (step 

15 383) and an image of the barcode is captured (step 384). hi a particular embodunent, the 
operator views the camera image on a scanning system display and to insure that the barcode 
is within the image region, and triggers the image capturing step by actuating an electronic 
switch, such as a trigger switch. 

[0074] The captured image is evaluated for correct exposure (step 385), and exposure 

20 parameters are adjusted (step 386) if necessary, after which a second image is captured (step 
387). An attempt to decode the second image is made (step 388), and an indication is given 
as to whether the scan is successful (step 389) or unsuccessful (step 390). Similarly, if the 
initial exposure was correct, an attempt to decode the first image is made (step 388), with 
similar indication of a successful or unsuccessful scan, 
25 [0075] If the screen is detected as being emissive, the illumination lamp remains off 

and an image is captured (step 391). This image is then evaluated for correct exposure (step 
392) and image parameters are adjusted (step 393) if necessary, and a second image is 
captured (step 394). An attempt to decode the second image is made (step 395), and if the 
scan is successfiil, a successful scan indication is generated (step 389). If the scan is 
30 unsuccessful, the display screen is evaluated for flicker (step 396). If no flicker is detected, a 
failed scan indicator is generated (step 390). Similarly, if the first image is within acceptable 
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exposure limits, an attempt is made to decode the first image, and if successful, a successful 
scan indication is generated, and if the decode is unsuccessful, the process flow proceeds to 
check for flicker 

[0076] If flicker is detected, the phase and period of the flicker is determined (step 

5 397), and the exposure time is adjusted according to the measured display flicker rate (step 
398). Another image is captured (step 399) and an attempt is made to decode this image (step 
400). If the decode is successful a successful scan indicator is generated (step 389), and if 
not, a failed scan indicator is generated (step 390). 

[0077] Thus, it is possible to scan barcodes, including two-dimensional barcodes, 

10 from a variety of electronic displays. The screen type is determined, and appropriate 

scanning conditions established. If the screen is emissive, the flicker is measured and used to 
set the exposure time of the imager. The scanner adapts to non-square barcode elements on a 
display with rectangular pixels by measuring known barcode elements and scaling the image 
data to produce barcode elements within the aspect ratio tolerance. The scarmer also reliably 
1 5 scans images from color displays by incorporating features in the components, such as a 
photopic camera, and in the software, such as digital filtering to remove interference patterns. 
[0078] While the present invention has been described in terms of the specific 

embodiments discussed in detail above, alternatives and equivalents may be apparent or 
become apparent to those skilled in the art without departing from the invention. For 
20 example, although Morie patterns have been used to describe interference, other types of 
interference might arise. Accordingly, the invention is to be interpreted according to the 
following claims. 
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